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Background. As the population ages, practice and policy need to be guided by accurate estimates 
of chronic disease burden in primary care. 

Objective. To produce a preliminary set of methodological considerations for cross-sectional and 
retrospective cohort studies of multi-morbidity in primary care using three studies as examples. 
Prevalence rate results from the three studies were re-estimated using identical age-sex groups. 

Methods. We compared the methods and results of three separate studies in primary care: (i) 
patients in the Saguenay region of Quebec, Canada (2005); (ii) a substudy of the BEACH (Bettering 
the Evaluation and Care of Health) programme in Australia (2008); and (iii) the DELPHI (De/iver 
Primary Health Care /nformation) project in South-western Ontario, Canada (2009). Areas where 
the methods of multi-morbidity studies may differ were identified. The percentage of patients with 
two or more chronic conditions was compared by age-sex groups. 

Results. Multi-morbidity prevalence varied by as much as 61%, where reported prevalence was 
95% among females aged 45-64 in the Saguenay study, 46% in the BEACH substudy and 34% in 
the DELPHI study. Several aspects of the methods and study designs were identified as differing 
among the studies, including the sampling of frequent attenders, sampling period, source of data, 
and both the definition and count of chronic conditions. 

Conclusions. Understanding the differences among the methods used to produce prevalence data 
on multi-morbidity in primary care can help explain the varying results. Standardization of meth- 
ods would allow for more valid inter-study comparisons. 
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Introduction 

In family practice, the care of patients with multiple 
health problems is a growing concern. New approaches 
to care must be developed to cope with what most pro- 
fessionals consider will be a tsunami of demands, given 
the aging population. 1 

Family medicine research has been woefully inad- 
equate in providing basic data on multi-morbidity that 
mimic the kinds of data readily available for common 
single chronic and infectious diseases, i.e. studies on 
prevalence, characterization, risk factors (or determi- 
nants) and prognosis. 2,3 

The purpose of this study is to assist researchers 
in their endeavours to launch prevalence studies of 



multi-morbidity and to help readers interpret and 
compare results from these studies. The need for 
such assistance became clear to the authors during 
the discussion after a presentation at the 2009 North 
American Primary Care Research Group. 4 In an 
attempt to put one study into context, the presenter 
briefly compared her results with those of several 
other studies. The authors of those compared studies 
happened to be in the room and explained, during the 
discussion period, which methods used in their studies 
may have led to the contrasting results. We quickly 
realized that (i) the 'Methods' sections of our articles 
could have been more detailed had the word limits not 
applied, leaving the reader unable to fairly interpret 
the results when comparing one study with another and 
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(ii) standards for calculating and reporting multi- 
morbidity were lacking. 

Therefore, our comparative analysis produced the 
following: (i) a preliminary set of methodological 
considerations for cross-sectional and retrospective 
cohort studies of multi-morbidity in primary care; (ii) 
a description of the methods across the three stud- 
ies as examples; (iii) re-estimates of prevalence rates 
conducted using identical age-sex groups; and (iv) a 
detailed assessment of the methods thought to be per- 
tinent to the rates. 



Methods 

This study reviewed the methods and results of three 
separate studies of multi-morbidity in primary care 
herein referred to as (i) the Saguenay study, from the 
Saguenay region of Quebec, Canada; 5 (ii) a BEACH 
substudy, from the Bettering the Evaluation and 
Care of Health programme in Australia; 67 and (iii) 
the DELPHI study, from the Deliver Primary //ealth 
Care /nformation Project in South-western Ontario, 
Canada. 4 - 8 The three studies were selected based on the 
participation of the authors in the discussion after the 
presentation at the 2009 North American Primary Care 
Research Group 4 regarding their varying results. 

For the purposes of this study, the raw data used in 
each of the three prior studies were extracted and the 
rates of multi-morbidity were compared for the rea- 
ligned six age-sex groups: 18-44 years, 45-64 years and 
>65 years. These data were not adjusted for the cluster 
design of their studies (clustering of patients by physi- 
cian or practice) or for representation of their respec- 
tive population profiles. 

In addition, the authors reviewed the three studies 
and emailed for clarification of the methods used to 
produce the published results. Points of difference 
among the methods were found in an iterative process, 
through review of the studies, by discussion at meetings 
and through teleconferences and email. The following 
areas were discussed: (i) research design; (ii) population 
and sampling; (iii) data collection, including definition of 
chronic active problems and list of chronic conditions; and 
(iv) the definition of multi- morbidity. All three studies had 
ethical approval by their respective institutions. 

Results 

Preliminary set of methodological considerations 
Table 1 presents the list of methodological considera- 
tions that arose from the detailed discussions of the 
authors. The methodological considerations appear on 
the left and the possible variations appear on the right. 
The judgement of which variations are stronger than 
others is presented in the Discussion. 



Table 2 presents the methods of the three studies of the 
authors in terms of the methodological considerations 
outlined above as a test of the preliminary list. During the 
completion of this table, the authors identified methodo- 
logical details that were not described in their publications 
or were described elsewhere, as in the case of the BEACH 
substudy 6 - 7 and the DELPHI study. 4 8 

Re-estimated prevalence rates 

Figure 1 shows the re-estimated prevalence of multi- 
morbidity for each of the three studies, stratified by 
age and sex. Prevalence in the Saguenay study was 
significantly higher than that found in the other stud- 
ies in all age-sex groups. Among the age groups, multi- 
morbidity prevalence varied by as much as 61 % , where 
the reported prevalence among women aged 45-64 
was 95% in the Saguenay study, 46% in the BEACH 
substudy and 34% in the DELPHI study. For men and 
women aged 18-44, prevalence did not differ between 
the BEACH and DELPHI studies. However, for both 
men and women aged 44-64 and >65, the prevalence 
varied significantly among the three studies. 

Discussion 

One interpretation of the results seen in Figure 1 is 
that they are real; the rates of multi-morbidity in the 
Saguenay region of Quebec are much higher than 
those in the region of South-western Ontario, with the 
rates in Australia, for the most part, falling in between. 
However, the authors suspect that an alternative inter- 
pretation holds: differences in method may play an 
important part and hence the emphasis, in this study, on 
parsing the details of the methods and weighing their 
importance to the rates found. 

There are at least three methodological issues raised 
by this comparison. 

The first issue that researchers should consider is 
the extent to which the studies are sampled differently. 
Sample size will make a difference to the rates of multi- 
morbidity, with larger samples, all other methods being 
equal, resulting in more reliable estimates. The BEACH 
substudy had both a larger sample and was of a national 
nature compared with the Saguenay and DELPHI studies, 
which represented small geographic regions. With regard 
to recruitment of patients within each practice, all three 
studies in the current comparison sampled visitors to the 
family practice. The time period set aside for recruitment 
of patients in each practice varied from 2-3 weeks in the 
Saguenay study to several days in the BEACH substudy 
and 6-12 months in the DELPHI study. Regardless of this 
time period, all studies were more likely to have sampled 
frequent attenders. 

Another aspect of sampling was that only the 
DELPHI study selected a subset of consecutive 
patients at random, to control the workload of coding 
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Table 1 Preliminary set of methodological considerations for cross-sectional and retrospective cohort studies in multi-morbidity (Methods 

Crystals for MM) 



Major categories Considerations Details Variations 



1. Design Research design - Cross-sectional study, retrospective cohort study 

2. Population and sampling Location - Country and/or region 

Sampling Sampling method e.g. Two-stage cluster sampling of FPs and then 

patients within them 

Primary care setting(s) Sampling frame e.g. All in region, affiliated with an academic 

institution and other subgroupings 
Selection method Random, non-random 

Sample size # Settings recruited or represented. 

Family practitioners (FPs) Sampling frame e.g. all FPs in the region or only those meeting 

specific criteria 
Selection method Random, non-random 

Sample size Number of FPs recruited 

Patients Sampling frame All citizens, total patient list and visiting 

patients 

Selection method Random, non-random and consecutive 

Sample size Number of patients contributing data 

Rationale for sample size - e.g. Based on the power of the test, when 

every patient had been approached or other 
considerations. 

3. Data and definition Data collection Source of data Whole medical record, specific time period in 

record, provider documentation at visit, pro- 
vider knowledge of patient, patient self-report 
Method of data collection Chart audit, electronic medical record data 

extract, questionnaire, physician form or other 
sources. 

Coding Morbidity coding No coding, nomenclature ICPC-Plus, coding 

with ordering principle 1CPC, or other coding 
methods 

Time Time period of data source Length of medical record, specific time period 

in medical record or at encounter 
Length of recruitment period Number of days, weeks, months? 
for patients for each FP 

Dates of data collection When did the researchers collect the data? 

Morbidity time focus Current conditions under medical management, 

within a specific time period or lifetime 
Definitions Definition of multi-morbidity Two or more conditions, three or more condi- 

tions or other definition 
Definition of chronic How were conditions identified as chronic and 

conditions was there a specific list of conditions used? 

Operational definition of the Did the FP or nurse use judgement to code 
count of chronic conditions evolving diagnoses as one condition? Could 

double counting occur? 

4. Outcomes Results Outcomes reported Prevalence of single morbidity, multiple morbid- 

ity, other health or health system outcomes (e.g. 
self-reported health, utilization of services) 
Confounders controlled e.g. Socio-economic status, mental health 

problems and other confounders studied must 
be specified 

Results presented e.g. Prevalence of multi-morbidity 



by the family physician (FP) and his or her staff. For 
the BEACH substudy, 30 consecutive patients were 
selected for recording from each FP's 100 patients con- 
tributing data to the larger BEACH Programme. In 
Saguenay, the study group was selected after limiting 
the time period to days or weeks. We believe that the 
random sample versus a time-limited consecutive sam- 
ple was not likely to affect the rate of multi-morbidity. 



The second methodological issue is the completeness 
and accuracy of the source of data and the period of time 
over which data were considered for each patient. The 
three studies used different sources of data: Saguenay 
used the medical record; the BEACH substudy used 
the FP report, patient report and medical record; and 
the DELPHI study used morbidity managed at the ini- 
tial and subsequent encounters coded by FPs. 
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Figure 1 Results of thirty-six post-hoc multi-morbidity tests by age and sex were significantly different among all three studies 

(Pearson chi-square analysis). 



Incompleteness may have affected each of the stud- 
ies in slightly different ways. When care was delivered 
by the FP but not written in the chart, there was inscrip- 
tion bias, leaving paper charts incomplete regarding 
psychological problems, social problems and some 
physical problems. This type of error could reflect the 
level of commitment of the FPs, their workload that 
day or recording fatigue. All diagnoses may not have 
been recorded. As an example, the prevalence of obe- 
sity in the Saguenay study was -4%, which was much 
lower than the prevalence in the community at large. 
If FPs were not actively managing obesity as an entity, 
they did not write it in their record. 14 Furthermore, the 
likelihood of incompleteness may be actually higher in 
studies of multi-morbidity than those of single morbid- 
ity because of the complexity of the patients' constella- 
tion of problems. Furthermore, the number of visits or 
encounters may have affected the completeness of the 
data abstracted. 

Another aspect of incompleteness arose from the 
time period over which the patient record was exam- 
ined. The Saguenay study used the entire record, which 
could have extended back over decades, to ascertain 
the number of morbidities. Similarly, the substudy of 
BEACH included FP's knowledge of the entire patient 
history and the patient's knowledge. In contrast, the 
DELPHI study used only encounters that occurred 
during a specific 2-year period. For identification in 
the DELPHI study, the FP had to have been actively 



managing and coding the chronic conditions dur- 
ing encounters over the specified 2 years. This impor- 
tant difference from the other two studies, in which 
the entire record including the problem list was used, 
leads to the possibility of an underestimate of multi- 
morbidity in the DELPHI study. 

Several other aspects of inaccuracy could have arisen. 
For example, a tentative diagnosis may have evolved 
over time and may or may not have been counted as 
multiple conditions. In the DELPHI study, such diagno- 
ses were most likely counted multiple times, leading to 
a possibility of an overestimate. In the Saguenay, such 
tentative diagnoses could have been counted many 
times or not, depending on the rigour of the reviewer 
in following and interpreting such illness trajectories. In 
the BEACH substudy, the problem could only be ticked 
(and therefore counted) once. 

The previous three paragraphs indicate that although 
all three studies were based on real-life practice, which 
was an asset in itself, the pressures of the FP's workload 
could have minimized the completeness in each of the 
three studies through different mechanisms. 

To summarize the issues of completeness and accu- 
racy, each methodology had strengths and limitations. 
Recall bias was a potential problem in the BEACH 
substudy, because FPs were asked to provide the mor- 
bidity data. To minimize recall bias, the study protocol 
asked FPs to ask the patient and to review the medi- 
cal chart as they completed the data form; this review 
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may have been conducted to a greater or lesser extent. 
Nonetheless, on the positive side, the FP had just seen 
the patient that day and re-familiarized him- or her- 
self with the patient's problems. Additionally, on the 
plus side was the fact that they were led to respond to 
specific conditions, not record problems in free text. 
Incompleteness of data may have been an issue in the 
Saguenay study if some chronic condition data had not 
yet been recorded in the medical chart in a place where 
reviewers were looking or perhaps not yet charted at 
all. Incompleteness of data may have been a problem 
in the DELPHI study also because FPs were asked to 
code all problems presenting for care at each encounter 
and they may have left out some conditions of multi- 
morbid patients due to coding fatigue; perhaps the first 
and second conditions may have been coded, but the 
third, fourth and fifth conditions may have been missed. 
Moreover, the patient may have ongoing problems 
that the FP did not manage at these encounters (e.g. 
because the patient was under specialist care for those 
problems). 

On the third issue, we noticed that the greatest dif- 
ference among the studies was the definition and the 
count of the chronic conditions. There were two dimen- 
sions of this issue: (i) the definition of a chronic con- 
dition; and (ii) how they were counted. The BEACH 
substudy captured chronic conditions out of a list of 18 
specific disease entities, plus broader diagnostic group 
options such as 'other cardiovascular', 'other psycholog- 
ical problem', 'other arthritis', 'cerebrovascular disease', 
'peripheral vascular disease' and 'malignant neoplasm'. 7 
The DELPHI project defined a chronic condition as 
any of a list of International Classification of Primary 
Care (ICPC-2-R) 11 rubrics adapted from Lamberts and 
Okkes. 15 The Saguenay study definition was open to all 
diagnoses; although they used a Quebec survey list 16 to 
categorize conditions, they added any conditions that 
did not appear on that survey list and extended their 
codebook accordingly. 

With regard to counting, the BEACH substudy clus- 
tered 22 diagnoses into 8 of the CIRS domains (plus 
another domain for malignant neoplasms) 12 ; therefore, 
the maximum number of chronic domains could not 
exceed nine. The conditions included those determined 
by the Australian Government as National Health 
Priority Areas, 17 selected on the basis of chronicity 18 
and management frequency in Australian general prac- 
tice. 9 The DELPHI study counted any of the 98 indi- 
vidual chronic conditions on the adapted Lamberts and 
Okkes list, 15 which meant that the maximum could not 
exceed 98. The Saguenay study counted all conditions 
and their unique codes actually tallied to 33 separate 
conditions. As an example, if a patient had hyperten- 
sion, peripheral vascular disease and other cardiovas- 
cular disease that would have counted as one disease 
domain in the BEACH substudy (vascular) and three 
conditions in the Saguenay and DELPHI studies. 



Similarly, osteoarthritis and low back pain (not related) 
would have been counted as one in the BEACH sub- 
study and as two in the Saguenay and DELPHI stud- 
ies. When comparing the prevalence in the sample of 
multi-morbidity, counting each morbidity type listed 
as one (rather than grouping the presence of one or 
multiple diseases into domains), the published results 
of the BEACH substudy 6 are far closer to those of the 
Saguenay estimates than the DELPHI estimates. 

To highlight the important methods issue, sampling 
of visiting patients may overestimate multi-morbidity 
but one can adjust for attendance rates. 14 The sources of 
data used in these three studies overall may have under- 
estimated multi-morbidity due to incompleteness of 
recording; however, some aspects of the data source for 
one study may have led to an overestimate. The largest 
difference among the three studies, likely to impact the 
rates of multi-morbidity, was the definition and count- 
ing of chronic conditions. Therefore, significant meth- 
odological issues may affect the results of studies of 
the prevalence of multi-morbidity, including sampling 
issues, source of data and definition and counting issues. 
We recommend therefore that international guideposts 
be developed for researchers to follow in an effort to 
create internationally comparable methods to study 
multi-morbidity. 



Conclusions 

Our goal is to improve methods of studying multi- 
morbidity. One option would be, instead of obtaining 
data from existing records, researchers could consider 
creating a data collection specifically designed for such 
studies. Such a tool could be used for billing purposes, 
so FPs could be reimbursed for the complexity level of 
patients. Such a tool for research and reimbursement 
will become more important as the population ages. 

However, in the absence of such a universal tool, what 
can researchers do to improve the status quo? Many of 
the methodological details reviewed by this compari- 
son would not be found in a 'Methods' section of typical 
articles, providing the reader of such studies with lim- 
ited ability to interpret the results or with the necessity 
to contact the author for further details. Therefore, we 
recommend that writers and readers insist on detailed 
descriptions of the type of sampling, the completeness 
and accuracy of the source of data and the definition of 
the chronic conditions. We offer Table 1 as a guide and 
call it 'Methods Crystals for MM'. 

Finally, the difference in definitions of chronic condi- 
tions was the most likely candidate to explain the large 
differences in rates of multi-morbidity among the three 
studies. This is a testable hypothesis. We recommend con- 
ducting further comparisons among multi-morbidity data 
using agreed-upon standards for the definition of chronic 
conditions and the way to count multi-morbidity, in order 
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to assess the impact of these methodological variations 
and enhance our understanding of multi-morbidity. 
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